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Background 

■  Validating  environmental  data  for  1 1  years 

■  Nationwide  client  base 

■  Government  agencies 

■  Native  American  tribal  governments 

■  Environmental  engineering  firms 
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AQA  Validates  Data  from  Numerous  Laboratories 

■  From  small  labs  to  the  largest  environmental 
laboratories 

■  We  have  observed  a  general  and  substantial  decline 
in  data  quality 

■  “QC  only”  DV,  manual  or  automated,  will  not  detect 
most  of  the  deficiencies 

■  These  deficiencies  are  often  detrimental  to  data 
accuracy  and  defensibility 
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Examples 

(We  didn ’t  have  too  look  hard 

to  find  these) 
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Falsification  of  Instrument  Calibration 

■  Laboratory  manually  changed  the  standard  areas 

■  Years  of  data  affected  on  high-profile  government 
project 

■  This  problem  was  found  by  manually  calculating  the 
curve  statistics  from  raw  data 

■  Would  not  have  been  caught  by  automated  data  review 
or  “QC  only”  data  validation 


Your  Environmental  Data  Qualify  Experts 


A^A 

Response  to  Average  RF  CAL  Failure: 

Hide  Behind  a  Poor  Linear  Fit 

■  Average  RF  calibration  failed 

■  Resort  to  a  linear  fit  (non-weighted) 

■  Coefficient  of  determination  (r2)  passed 

■  However,  concentration-intercept  was  unacceptably 
different  from  zero  and  negative  (see  next  slide) 

■  Both  positive  and  negative  intercept  situations  often 
observed.  Labs  only  look  at  r2  value  and  not  the 
intercept. 
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Small  Concentrations  Missed  /  Higher 
Concentrations  Biased  Low 


Your  Environmental  Data  Qualify  Experts 


Analytical  Quality 


Associates,  Inc. 


Prohibited  Removal  of  Calibration  Points 

■  Removal  of  interior  calibration  points  for  acetone 

■  Direct  violation  of  EPA  Method  8000 

■  Direct  violation  of  laboratory  SOPs  and  ethics 
agreements 

■  All  results  acquired  against  this  curve  are  grossly 
indefensible 
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97%  Failure  Rate  for  MS/MSD  Analyses 

■  Method  8151 

■  Laboratory  claimed  they  had  adequate 
cleanup  (GPC)  capabilities 

■  No  GPC  existed 

■  No  confidence  in  the  reliability  of  the 
data 
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Analyzing  QC  on  a  Separate  Instrument 


■  Defeats  the  several  key  goals  of  QC.  For  example,  MB 
analysis  will  indicate  contamination  of  prep  and  instrument. 
LCS  is  used  to  measure  control  of  entire  analytical  process, 
not  just  extraction. 

■  Laboratory  returned  to  same  practice  after  being  notified  it 
was  not  acceptable 

■  Whether  a  time-saver  or  malicious,  severely  hampers 
defensibility  of  data. 

■  Dramatically  increases  data  validation  labor  and  cc 
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Some  Other  Examples 

•  Unreported  positive  hits 

•  Misreported  QC  recoveries 

•  Multiple  manual  integrations  to  make  QC  pass 

•  Misused/overused  “matrix  effects”  assertions 

•  Failure  to  perform  required  re-analyses 

•  Rampant  QC/IS/surrogate  failures 

•  Failure  to  maintain  certifications 

•  Failure  to  perform  blank  population  MDCs 
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What’s  Causing  This  Decline? 


Largely  Attributable  to  Lowest  Price  —  Technically 

Acceptable  (LPTA)  Contracting 
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Problems  with  LPT  A 

•  Failure  to  adequately  determine  Technical  Acceptance 

•  Deviation  from  Best  Value  approach  to  contracting 

•  Price  becomes  only  concern,  there  is  a  perfunctory  nod  to 
actual  quality,  not  a  real  inquiry 
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Effects  of  LPTA  on  Laboratory  Operations  (1) 

■  You  are  not  the  lowest  bidder,  you  lose! 

■  You  must  trim  laboratory  operations  so  you  can  be  the 
low  bidder 
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Effects  ofLPTA  on  Laboratory  Operations  (2) 

■  Increased  laboratory  burden 

■  Decreased  per  sample  price  /  increased  number  of 
samples  to  generate  same  revenue 

■  Analytical  shortcuts 

■  Eroded  management  commitment  to  quality 

■  Decreased  quality  assurance  oversight 

■  Deliverable  corrections  and  CAR  responses  not 
delivered  in  a  timely  manner 
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What  is  Technically  Acceptable? 

■  Laboratory  scope  and  capacity 

■  Proper  equipment 

■  Qualified  personnel 

■  Quality  systems 

■  Reporting  capability 

■  Track  record  of  compliant  on-time  performance 
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How  is  Technical  Acceptability  Typically  Verified? 

■  Review  of  laboratory  proposals  only 
(no  pre-qualification  audit) 

■  Pre-qualification  audits  by  QA  officers  only 

■  Sending  technical  staff  who  are  not  experienced 
analytical  chemists 

■  Reliable  verification  is  accomplished  via  a  pre¬ 
qualification  audit  performed  by  experienced  analytical 
chemist(s) 
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Common  Misconceptions 

■  Laboratory  certifications  guarantee  a  technically 
qualified  lab 

■  Implementation  of  a  standard  is  only  as  good  as  the 
personnel  implementing  it 

■  AQA  has  observed  serious  failures  by  certified 
laboratories 

■  Information  supplied  by  laboratories  is  comprehensive 
and  accurate 

■  Laboratory  assertions  are  often  misleading  at 
best 
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Consequences  of  Inadequate  Verification 

■  Acquisition  of  indefensible  data 

■  Misleading  data  such  as  false  positives  or  false 
negatives 

■  Diminished  quality  (excessive  qualification) 

■  Contracts  awarded  to  laboratories  that  do  not  have 
sufficient  capacity  to  generate  compliant  data  on-time 
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Summary 

■  Overall  decline  in  environmental  laboratory  data  quality 

■  LPTA  contracting,  to  a  substantial  degree,  is  a  root 
cause  of  the  declining  quality 

■  Meaningful  technical  acceptability  verification  needed 

■  Problems  presented  are  the  tip  of  the  iceberg 

■  Problems  won’t  be  found  using  automated  data 
validation  software  or  cursory  “QC  Only”  data  validation 
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